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Abstract 
UPSTART is a federally funded 13 validation project that uses a computer-based program to 
develop the school readiness skills of preschool children in rural Utah. Researchers used a 
randomized control trial design to evaluate the impact of the program in advancing children’s 
early literacy skills. Preschoolers in the experimental group were randomly assigned to the 
UPSTART Reading software, while control group students were assigned to UPSTART Math. 
Standardized early literacy assessments were administered prior to program commencement and 
upon completion. Results revealed that there was a significant difference in children’s mean 
scores on measures of letter knowledge and phonological awareness, after controlling for prior 
knowledge, missing pre-test data, and children’s school district between those who participated 
in UPSTART Reading and those in the comparison group. There were no differences between 
the two groups on assessments measuring vocabulary and oral language or listening 


comprehension. 


Impact of UPSTART Reading Participation for Rural Students While Controlling 


for Prior Reading Achievement and School District 


The pathway to fluid reading begins with the development of preliteracy skills, such as 
concepts of print, alphabet knowledge, and phonological awareness, and progresses to reading 
words and text comprehension. A key precursor of literacy acquisition is phonological 
awareness, the ability to recognize, identify, and manipulate the smaller sound units within 
words, independent of their meaning (Cassady & Smith, 2004). Another crucial early literacy 
skill is letter knowledge, or the knowledge of the names and sounds associated with printed 
letters. 

Research has consistently linked these early literacy skills with later reading achievement 
in school. Emergent readers who demonstrated phonemic blending and segmenting skills were 
found to be significantly more likely to successfully acquire beginning reading skills 
(Bauserman, Cassady, Smith, & Stroud, 2005). The National Early Literacy Panel (NELP) 
conducted a large-scale meta-analysis of early literacy research and found that emergent literacy 
skills such as phonological awareness and alphabet knowledge had a clear and consistently 
strong relationship with later reading skills, in addition to conventional literacy skills such as 
decoding, oral reading fluency, and comprehension that were also predictors of later achievement 
(NELP, 2008). Similarly, an analysis of six longitudinal datasets indicated that early language 
skills such as vocabulary, letter knowledge, and word sounds were a consistent predictor of later 
reading achievement, along with math and attention skills (Duncan et al., 2007). 

Students who enter kindergarten with early literacy skills and are ready to learn get better 


grades, are more likely to graduate from high school, and have a greater chance of entering 


successful careers as adults than children who are less ready when they begin kindergarten 
(Reynolds et al., 2007). Conversely, young children who struggle with reading in early 
elementary school tend to fall behind their peers and perform at lower levels later in school. 

High-quality early education settings that use age-appropriate curricula with clearly 
articulated goals can contribute to improvements in academic areas such as literacy and 
numeracy so that children have the skills for academic success at school entry (Phillips et al., 
2017). While there is evidence that children from all socioeconomic backgrounds can benefit 
from early education programs, program effects tend to be higher and learning improvement 
greater for socioeconomically disadvantaged children (Barnett, 2008; Benner, Thornton, & 
Crosnoe, 2017; Phillips et al., 2017). 

Although a large body of research points to the benefits of high-quality early education in 
fostering children’s cognitive and emotional skills in preparation for the increasing rigor of 
formal education (Bakken, Brown, & Downing, 2017; Council of Economic Advisers, 2015; 
Duncan & Magnuson, 2013; Ramey & Ramey, 2004; Yoshikawa et al., 2013), children living in 
rural communities are less likely to participate in formal early education programs than children 
living in urban areas (Temple, 2009). Children reared in rural environments are more likely to 
live in lower income families compared to children living in urban environments (Addy, 
Englehardt, & Skinner, 2013) and may have limited access to high-quality preschools (Khan, 
Justice, & Jiang, 2016). Early education opportunities can prepare children for school, but 
researchers report that about 15% of children in rural communities attend a high-quality pre- 
kindergarten program, compared to 30% of children in urban and suburban areas (Nores & 
Barnett, 2014). Such disparities in early childhood education opportunities for rural children 


may be directly connected to recent findings from the Early Childhood Longitudinal Study that 


show that kindergarteners in rural communities perform more poorly than suburban and urban 
kindergartners on measures of math and reading (Miller & Votruba-Drzal, 2013). 
Education Technology for the Underserved 

Education technology offers an opportunity for rural children to receive early education 
instruction even in the face of limited center-based offerings or remote geographical locations. 
Technology is becoming a regular part of everyday life for many young learners and its use is 
increasing as a social and educational phenomenon (Laidlaw & Wong, 2016). Two-thirds of 
preschool-aged children actively engage with computer related technologies (National Center for 
Educational Statistics, 2005) and access to the internet in educational settings increased from 
51% to 94% from 1998 to 2005 (NCES, 2008). 

However, in spite of the adoption of new technologies such as smartphones and personal 
tablets, there is a persistent digital divide between urban and rural users, reminiscent of the 
disparity between rural and urban children’s participation in early education settings. In their 
work with the National Telecommunications and Information Administration (NTIA), Carlson 
and Gross (2016) report that 69% of rural residents in 2015 reported internet use compared to 
75% of urban residents. Moreover, Carlson and Gross note that the digital divide between urban 
and rural residents was evident with rural families with low incomes, with 66% of rural residents 
with family incomes between $25,000 and $49,999 reporting internet use, compared to 70% of 
their urban counterparts with similar income levels. 

Recognizing these disparities, broadband grant programs funded by the American 
Recovery and Reinvestment Act of 2009 invested approximately $4 billion in projects to support 
and enhance broadband infrastructure throughout the United States and promote statewide 


broadband planning (NTIA, 2018). With the increasing adoption of widespread broadband 


connectivity, educational technologies can bridge the gap in high quality pre-kindergarten 
programs in rural communities where access to early learning curriculum is hindered by a lack of 
resources and geographical barriers (Smith, Patterson, & Doggett, 2008). To that end, there has 
been an abundance of educational software programs targeted to young learners, child care 
centers, parents, and schools to support the development of children’s early literacy and reading 
skills (Wood, et al., 2012) 

To date, positive findings for the use of reading software have been reported in the 
literature, creating a growing evidence base that children’s lives can benefit from educational 
technology. A review of forty-two studies of computer-assisted instruction with beginning 
readers showed a positive overall effect size (d = .19) (Block, Oostsdam, Otter, & Overmat, 
2002). Similarly, kindergarten students receiving computer-assisted instruction in phonological 
awareness and letter-sound correspondence had significantly higher scores on post-test measures 
of phonological awareness than children who did not receive computer instruction (Macaruso & 
Walker, 2011). Work by Abrami, Borohkovski, and Lysenko (2015) demonstrated that a 
computer-based reading program had significantly positive effects on children’s phonological 
awareness, listening comprehension, and vocabulary knowledge. There was a positive 
relationship between exposure to educational technology curriculum during the course of the 
preschool day on children’s school readiness, and these benefits were enhanced by children’s 
additional computer experience at home (Li, Atkins, Stanton, 2006). 

Although sporadic research on educational technology programs has been conducted in 
the past, researchers describe limited formal evaluation on software programs prior to release or 
once they are available to the general public (Grant et al., 2012) and when research is conducted, 


it often fails to meet methodological standards for high-quality research such as using adequate 


comparison groups, including implementation results, or providing the necessary data to 
determine effect sizes (Murphy, Penuel, Means, Korback, Whaley, & Allen, 2002). 
Moreover, although there are numerous educational software programs with the objective of 
helping children learn how to read, there is no regulatory body that screens software programs 
for educational value or quality (Willouhby & Wood, 2008). 

Noting these trends in educational technology and the need for rigorous research to 
measure its impact, the U.S. Department of Education identified educational programs that serve 
rural communities and instructional solutions that effectively use technology as two of the most 
pressing priorities for education researchers in the 2013 13 Notice of Final Priorities, 
Requirements, Definitions, and Selection Criteria. The Evaluation & Training Institute (ETI) 
partnered with the Department of Education and the Waterford Research Institute (13 grantee) to 
study the effects of the UPSTART program on the emerging literacy skills of children living in 


rural districts throughout Utah. 


Learning Technology Under Study 

UPSTART is a school readiness program that uses a home-based educational technology 
model with supportive resources that is well-suited to preschoolers and families residing in rural 
communities. The objective of UPSTART is to provide service to students in traditionally 
underserved rural areas, foster early cognitive growth, close reading achievement gaps, and 
increase school readiness at entry to kindergarten. 

The logic model shown in Figure | illustrates the key components of the Rural 
UPSTART project, the mediators through which the intervention is designed to work, and the 


short- and long-term outcomes. While the UPSTART adaptive educational software is the core 


component of the program, the provision of technology in the form of computer hardware and/or 
high-speed internet ensure equitable access for all. Additionally, UPSTART families are 
provided with technical, motivational, and curricular support to encourage program software 


usage and facilitate the academic supervision of children by parents. 


Key Components 


Inputs/Activities 


Figure | 


Rural UPSTART Project Logic Model 


Mediators 


Program Usage 


Usage requirements 
¢ 5 times/week 


¢ 15 minutes each session 
* 9 months of total use 


Changes in Student 
Knowledge 


Educational Technology 


* Increased scores on reading 
curriculum software 

* Increased mastery of 
curriculum content 


Provision of Technology 
¢ Distribute computers to 


families in need 

* Provide families in need with 
internet access 

¢ Establish access to cloud- 
based software 


Support 


Parent Support 
¢ Parent training session 


conducted by Waterford staff 
¢ Parent Manager Portal 
software feataure 


Curricular Support 
¢ Emails about curricular 


features of software program 


Motivational Support 
¢ Personal Care 


Representatives monitor 
program use and conduct 
motivational calls 

* District liaisons hold special 
events for preschoolers and 
motivate parents 


Changes in Student 
Attitudes 


* Increased engagement in 
reading 

* Increased self-efficacy in 
reading 

¢ Increased persistence with 
reading 

¢ Increased motivation to read 


—> 


Changes in Parent 
Attitudes/Knowledge 


children’s learning in reading 
¢ Increased understanding of 
the UPSTART software 

¢ Increased monitoring of 
children’s literacy skills 


* Increased efficacy to support 


Short-Term Outcomes 


Increases in skills associated 
with pre-literacy 

¢ Alphabet knowledge 

¢ Visual discrimination 

¢ Auditory discrimination 

« Phonemic awareness 

¢ Oral comprehension 

¢ Vocabulary 

¢ Expressive language 

¢ Phonological awareness 

¢ Pre-primer word recognition 
¢ Survival sight words 


Long-Term Outcomes 


¢ Increased rates of proficiency 
on standardized academic 
tests of reading and/or 
English/Language Arts 

¢ Increased rates of grade 
promotion 

¢ Fewer students with the need 
for reading intervention 
resources 

¢ Increased engagement and 
motivation to read 


The principal component of UPSTART is an in-home computer-based school readiness 
software program that provides preschool children with reading, math, and science curriculum, 
with a focus on reading instruction. The program is designed to promote mastery of literacy and 
math skills that will prepare young children for entry into school by providing an individualized 
learning experience through a lesson sequencer that adapts to each child’s skill level. Based on 
student performance, the sequencer will run remedial activities to reteach and practice skills 
again or advance to another objective if students are mastering concepts. Content is delivered 
online through lessons that adapt to a child’s skill level, using multimedia, digital books, songs, 
and other online activities. Recommended usage of the software is 15 minutes a day, 5 days a 
week. 

Based on early reading instruction guidelines outlined by the National Reading Panel 
(2008) that emphasize phonemic awareness, phonics, fluency, comprehension, and vocabulary, 
the UPSTART Reading program is designed to use research-based best practices for early 
literacy instruction. The UPSTART Reading curriculum seeks to lay the foundation for skilled 
reading by emphasizing precursor skills related to decoding and comprehension, two processes 
that are the hallmark of reading fluency (Hoover and Gough, 1990). Table 1 showcases the 
reading domains and skills taught by UPSTART Reading at the first level! of the curriculum: 


phonics, comprehension/vocabulary, language concepts, and phonological awareness. 


1 Level One is the beginning point of the curriculum where the preschool child begins as a nonreader and is introduced to skills 
designed to teach the child to read. Levels range from one to three and the child is tested at the beginning of the program and 
placed in a level based on his or her performance. 


Table 1: UPSTART Reading Program Domains and Skills 


UPSTART Reading Domains Level 1 Reading Skill 

Phonics e Recognize A through Z, and a through z 
Systematically builds from not reading to e Learn 10 letter sounds and 20 sight words to read 10 
confident reading at 90 words a minute leveled readers 


Spell child’s name 


Comprehension/Vocabulary 


Read along and understand nursery rhymes 


Develops vocabulary and critical e Read along and understand alliterative books 

thinking skills through rich reading e Learn 255 target vocabulary words 

experiences 

Language Concepts e Understand print (left-to-right, letters, pictures, 

Introduces concepts of written language words, text) 

(from letters and pictures to basic e Develop oral language skills (colors, shapes, 

grammar) numbers, sizes, etc.) 

Phonological Awareness e Break words into individual sounds (cat to (/k/ /a/ /t/) 

Develops awareness of individual sounds _ e Blend individual sounds into words (/k/ /a/ /t/ to cat) 

in words e Change a sound in a word to make a new word (cat to 
bat) 


Research Question and Hypothesis 
We designed a randomized control-trial (RCT) experiment to test our primary research 
question: 

Do preschool children randomly assigned to receive the UPSTART Reading software 
program for one year have higher scores than their counterparts assigned to the 
UPSTART Math/Science program on the following measures of emerging literacy: 

a. Pre-literacy discrimination 

b. Letter knowledge, 


c. Phonological awareness, 
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d. Decoding, 
e. Vocabulary and oral language, and 
f. Listening comprehension? 

We hypothesized that if the UPSTART Reading program has no effect on improving 
early literacy skills, then we would expect children who participated in UPSTART Reading 
(treatment group) to perform at the same level as children who received the Math/Science 
program (control group) on measures of early literacy. Conversely, if UPSTART Reading 
influences early literacy, then children in UPSTART Reading should perform significantly better 


than the comparison group on literacy outcomes. 


Methods 

Participants 

All eighteen rural school districts in the state of Utah participated in the UPSTART 
program. Within each school district the program was offered to all age-eligible English- 
speaking children. Five of the smaller districts (Daggett, Piute, Rich, Tintic, and Wayne) were 
excluded from the preschool study due to insufficient sample size and geographical constraints. 
All families who registered for the UPSTART program and resided in the 13 target school 
districts were contacted by ETI. During this initial interaction, we described the evaluation in 
detail, established whether or not the family was interested in participating, conducted a 
screening to determine evaluation eligibility, and if the family was eligible, assigned the parent 
and child to a testing session. Program families were excluded from the evaluation if their child 


did not speak English or had a diagnosed learning disability. 
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Participants were 491 preschoolers from 13 rural Utah school districts who enrolled in 
the UPSTART program, sampled from a total UPSTART program group size of approximately 
655 eligible families (roughly 75% of the total program group size at the start of testing.) 
Preschoolers were randomly assigned to the UPSTART Reading program (N = 252) and to 
UPSTART Math (NV = 239) within districts. 

Measures 

Evaluation participants were administered two early literacy instruments to obtain 
baseline data: the Literacy scale of the Academic Skills/Cognitive Development component of 
the Brigance Inventory of Early Development III (Brigance, 2013) and the Preschool Early 
Literacy Indicators (PELI) (Dynamic Measurement Group, 2014). Table 2 presents the early 
literacy domains of interest, along with the corresponding literacy subtest used to measure 


children’s abilities. 


Table 2: Literacy Domains by Instrument Subscale 


Early Literacy Domain Instrument Subscale 


Pre-literacy Brigance IED III Visual Discrimination 
Auditory Discrimination 
Letter knowledge Brigance IED III Recites Alphabet 
Names Uppercase Letters 


Phonological awareness Brigance IED III Phonological Awareness 

Phonemic Manipulation 

PELI Phonological Awareness 
(First Word Sounds/Parts) 

Decoding Brigance IED III Word Recognition 

Reads Words from Common Signs 
Vocabulary/Oral Language PELI Vocabulary and Oral Language 
Listening comprehension PELI Listening Comprehension 
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Children’s letter knowledge was assessed with the Names Uppercase Letters subtest from 
the Brigance, a 26-item scale that ranging from | to 26, and asks children to name all twenty-six 
letters of the alphabet. Phonological awareness was measured by the Phonemic Manipulation 
subscale of the Brigance (7-item scale ranging from 0 to 7) and the Phonological Awareness 
subscale of the PELI (10-item scale ranging from 0 to 15). While the Phonemic Manipulation 
subscale measures children’s abilities to blend phonemic sounds into whole words and segment 
words into separate parts, the Phonological Awareness subscale assesses children’s ability to 
identify the initial sounds and initial word parts of words. The Vocabulary and Oral Language 
subscale assessed children’s ability to name, describe, and identify key elements of a picture 
related to a theme and consisted of a 15-item scale with scores that ranged from 0 to 35. 
Children’s listening comprehension was measured with the PELI Listening Comprehension 
subtest (14-items, ranging from 0 to 23) that assessed children’s ability to make predictions and 
inferences, answer comprehension questions related to a simple story, and to orally fill in the 
missing words of a story summary. 

Design and Procedures 

An intent-to-treat randomized control trial design was used (diagrammed in Figure 2), 
and included pre- and post-program early literacy measures administered to a group of treatment 
students (UPSTART Reading) and a group control students (UPSTART Math), who were 


randomly assigned to their condition within school districts prior to using the program. 
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Figure 2: The Evaluation Design of the Preschool Study 


Summer Summer 
2014 2015 
PSTART Readi 
Pre-Test saa caine Post-Test 
Random Treatment : 
: Kindergarten 
Assignment UPSTART Math 
Pre-Test Post-Test 
Control 


Children were recruited to participate in the evaluation in the early summer of 2014. 
Children in both groups completed their respective programs over a nine-month period and post- 
program literacy assessments were conducted after the UPSTART program was completed in 
Summer 2015, just before the children entered kindergarten. 

Assessments were individually administered to children by trained test administrators 
who were unaware of children’s assignment to the treatment or control group and assessments 
were held at central locations within each district (e.g., elementary schools, libraries, community 
centers). The entire assessment procedure was completed in 30-40 minutes on average and was 
collected during the same period and with identical procedures for both groups. 

Data Analytic Approach 

We examined students’ baseline characteristics to confirm that treatment and control 
groups were equivalent at pre-test on factors that may influence emergent literacy skills 
measured at kindergarten, such as initial differences between the two groups (e.g., pre-test 
achievement scores), as well as demographic factors that may differentiate between the treatment 
and control groups and found no statistically significant differences. 

We begin by presenting pre-test and post-test mean scores by treatment group for each 
outcome measure in the Results section. Using independent samples ¢-tests, we established 


equivalence between the treatment and control group at pre-test, and determined if there are any 
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significant differences between the two groups on post-program outcome measures. Mean 
differences upon program completion (at post-test) are presented, along with ¢-values and 
significance levels. 

In order to control for factors that may influence our outcome variables of interest other 
than the treatment variable, we conducted a series of linear regressions for each outcome 
variable. Prior knowledge is an obvious control variable, and baseline data were collected before 
children began UPSTART Reading instruction on all outcome variables of interest. These pre- 
test measures were identical to the measures used to measure outcome data after a year of 
UPSTART Reading instruction was complete. Although preschoolers were randomly assigned to 
the treatment (UPSTART Reading) and control (UPSTART Math) conditions, assignment was 
within thirteen school district blocks (see Table 3). To adjust for unequal allocation to the 
treatment and control conditions within districts, we added dichotomous dummy variables that 
differentiated the district subsamples. 


Table 3: Number of Participants by School District 


District Reading Math Total 
Beaver 13 19 32 
Duchesne 39 35 74 
Emery 23 20 43 
Garfield 11 10 21 
Grand 9 9 18 
Kane 17 16 33 
Millard 21 15 36 
North Sanpete 25 23 48 
North Summit 5 7 12 
Sevier 20 22 42 
San Juan 14 13 27 
South Sanpete 32 29 61 
South Summit 23 21 44 
Total 252 239 49] 
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Missing Data 

In an ideal world, all children who were pre-tested would have been post-tested. 
However, as in most longitudinal studies that rely on repeated measures, that ideal is rarely 
attained. Pre-tests were administered to 549 children (285 assigned to the Reading treatment 
condition and 264 assigned to the control condition), and 491 students completed a post-test one 
year later (252 children enrolled in UPSTART Reading and 239 children enrolled in UPSTART 
Math). Based on these numbers, the study had an overall attrition rate of 11% and a differential 
attrition rate of 2% (see Table 2). According to standards set by What Works Clearinghouse 
(2017), an overall attrition rate of 11% must have a differential attrition rate of less than 6% to 
have a tolerable threat of bias under both optimistic (i.e., attrition is exogenous or unrelated to 
the intervention) and cautious (i.e., attrition is endogenous or related to the intervention) 
assumptions regarding the relationship between attrition and outcomes. Attrition rates in Table 4 
show that overall and differential attrition between treatment and control students was within 
acceptable levels of bias set by What Works Clearinghouse (2017), that threats from selective 
attrition were minimized (Miller & Hollist, 2007), and that our RCT was intact after post- 


program data collection. 


Table 4: Number of Participants in Evaluation Sample 


Group Reading Math Total Sample 

Children Pre-Tested 285 264 549 

Children Post-Test 252 239 491 

% Attrition 12% 10% 11% 
Statistical Analysis 


Following the recommendations outlined in What Works Clearinghouse (2017) and 


Puma, Olsen, Bell, and Price (2009), we set missing pre-test scores to a missing value constant 
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and added a missing data flag to the regression model. For cases with missing post-test data, we 
used case deletion to remove those cases from our analytic sample and marked the missing cases 
as attrition. 

With these considerations in mind, we define the following variables for each preschool 
student in linear regressions to estimate the impact of UPSTART reading on our outcome 
variables of interest: Yi; is the preschooler’s score on post-test literacy measures of letter 
knowledge, phonological awareness, oral language/vocabulary, and listening comprehension; 
Treatment (T;;) is an indicator for whether the preschooler received the intervention (Treatment = 
1 if the student was randomly assigned to the UPSTART Reading treatment intervention, 
Treatment = 0 if the preschooler was randomly assigned to the UPSTART Math control 
condition); aie is the preschooler’s score on pre-test literacy measures (pre-test covariate); 
yponerrs is an indicator for records with missing pre-test data; and Block; is a district block 
dummy covariate and indicates whether or not a preschooler resided in one of the thirteen 
specific school districts while participating in the UPSTART Reading and Math programs. One 


possible linear regression model that uses these variables is the following: 


J=1. 
Yij = Bo + B,(Ty) + BAe 5) + Bal yessingkre) + >: Vj Block, + €jj 


j=1 
The Bs in Eq. | are regression coefficients that describe the relationship between each variable 
and the preschooler’s post-test score: 
e ois the intercept; 
e 1 is the expected increase in the post-test score for preschoolers who participated in the 
UPSTART Reading intervention relative to students who did not receive the intervention. 


e £2 is the effect of pre-test data; 
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e {31s the effect of having missing data at pre-test; 

e £4 through Bis are fixed block dummy variables to measure district effects. 
Separate linear regressions were run to estimate the effects of our outcomes of interest: letter 
knowledge, phonological awareness (first word sounds and phoneme manipulation), 


vocabulary/oral language, listening comprehension. 


Results 

Preliminary Results. Table 5 presents pre-test and post-test mean scores on the early 
literacy constructs of interest: pre-literacy discrimination, letter knowledge, phonological 
awareness, decoding, vocabulary/oral language, and listening comprehension. Initial results from 
t-tests indicate that there were no significant pre-program differences between children assigned 
to treatment and control conditions on any subscale, indicating comparable levels of early 
literacy skills between the two experimental groups prior to beginning the UPSTART program. 
There were, however, significant post-program differences between children enrolled in 
UPSTART Reading and those participating in UPSTART Math on measures of letter knowledge 
[identifying uppercase letters (4(489) = 6.09, p = .000) and reciting the alphabet (#(489) = 2.86, p 
= .004)]; phonological awareness [phonological awareness (#(489) = 2.25, p = .025), phoneme 
manipulation (¢(489) = 3.58, p = .000), and initial word sounds (¢(489) = 3.20, p = .001)]; and 
decoding [word recognition (#(489) = 5.67, p = .000) and reading words from common signs 
(t(489) = 2.85, p = .005)], with UPSTART Reading children outperforming their Math 
counterparts. There was no significant difference between groups at post-test on measures of pre- 
literacy discrimination [visual discrimination (4(489) = .96, p = .339), auditory discrimination 
(t(489) = .37, p = .709)]; vocabulary and oral language (4(489) = -.37, p = .710); or listening 


comprehension (¢(489) = .70, p = .486). 


19 


Table 5: Subscale Pre-Test and Post-Test Means of Treatment-Control Groups by Literacy Construct 


Pre-test Post-Test 
Subscale N Mean SD t-value N Mean SD t-value 
Pre-literacy 
Visual Discrimination 
Reading 247 ~—s- 11.63 5.449 1.32 252 16.59 3.204 6 
Math 235 ~=—10.96 5.654 239 16.33 2.915 
Auditory Discrimination 
Reading 247 6.47 2.543 1.34 252 7.79 2.693 47 
Math 234 6.16 2.473 239 7.70 2.458 
Letter Knowledge 
Recites Alphabet 
Reading 246 7.51 9.051 252 19.39 9.074 
36 2.86** 
Math 230 7.23 8.302 239 16.91 10.132 
Identifies Uppercase Letters 
Reading 247 9.23 8.862 95 252 20.53 7.106 6.09** 
Math 233 8.61 9.099 ; 239 16.19 8.634 
Phonological Awareness 
Phonological Awareness 
Reading 248 4.49 2.484 133 252 6.57 2.444 2.25% 
Math 233 4.19 2.532 239 6.06 2.564 
Phoneme Manipulation 
Reading 246 2.46 1.339 Ad 252 4.07 1.529 45g 
Math 233 2.40 1.453 239 3.56 1.631 
Initial Word Sounds 
Reading 242 4.69 4.643 25 252 10.53 4.673 3.20** 
Math 232 4.79 4.813 239 9.19 4.592 
Decoding 
Word Recognition 
Reading 242 27 1.419 56 252 3.38 3.997 5.67%* 
Math 230 21 .793 239 1.50 3.288 
Reads Words from Signs 
Reading 244 1.09 1.360 61 252 2.25 1.951 > gst 
Math 231 1.02 872 239 1.75 1.969 
Vocabulary/Oral Language 
Vocabulary/Oral Language 
Reading 248 ~=16.96 6.851 18 252 22.48 6.197 37 
Math 234 ~=17.08 6.820 239 22.68 6.043 
Listening Comprehension 
Listening Comprehension 
Reading 245 = 14.97 4.403 1.06 252 18.36 3.642 70 
Math 232 14.53 4.729 239 18.14 3.429 
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Linear Regression Analysis 

Ordinary least squares multiple regression models were conducted to examine the 
relationship between the outcome variables of interest and potential predictors, including 
treatment status, baseline literacy scores at pre-test, district blocking variables to control for 
unequal allocation to the treatment and control conditions within districts, and a dummy variable 
adjustment to control for missing pre-test data. Summaries of results without the district blocking 
variables are presented by literacy construct. The full regression models are included in 
Appendix A. 

Pre-literacy discrimination. Two separate multiple regressions were conducted with 
visual discrimination and auditory discrimination scores as dependent variables and treatment 
group membership, pre-test baseline scores, school district, and missing pre-test data as predictor 
variables. Table 6 presents the regression results for visual discrimination and auditory 
discrimination, respectively, and reveal that participation in the treatment condition (UPSTART 
Reading) did not have a significant impact on children’s visual discrimination skills (6 = 0.02, p 
= .530) or auditory discrimination (f = 0.01, p = .828) 


Table 6: Regression Summary of Predictors of Pre-Literacy Discrimination 


Visual Discrimination Auditory Discrimination 
(N=482) (N=481) 

Variable B SE B pb B SE B Bb 
(Constant) 13.75 0.54 6.67 0.54 
Treatment 0.15 0.25 0.03 0.05 0.22 0.01 
Pre-Test 0.20 0.02 0.38 0.23 0.05 0.23 
Missing Pre-Test Data -0.95 0.88 -0.05 -3.48 0.81 -0.19 
R 0.22 0.14 
F 8.85 5.00 


* p<0.05; ** p<0.01 
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Letter Knowledge. To test the hypothesis that participation in UPSTART Reading had a 
unique positive impact on preschoolers’ letter knowledge controlling for prior knowledge, school 
district, and missing pre-test data, a regression analysis was performed. Results of the regression 
analyses displayed in Tables 7 and 8 provided confirmation that UPSTART Reading had a 
positive influence on children’s alphabet knowledge. The ability to identify uppercase letters had 
a significant beta coefficient for the treatment predictor (f = .25, p = .000), indicating that 
participation in UPSTART had a significant impact on children’s letter knowledge while 
controlling for prior knowledge, missing pre-test data, and school district. On the second letter 
knowledge measure that assessed children’s capacity to recite the alphabet, assignment to the 


treatment condition was a small, but significant predictor of children’s scores (f = .10, p = .017). 


Table 7: Regression Analysis of Predictors of Letter Knowledge 


Uppercase Letter 
Knowledge Recites Alphabet 
(N=480) (N=476) 
Variable B SE B B B SE B B 
(Constant) 10.80 1.13 11.25 1.73 
Treatment 4.04 0.54 0.25** 1.97 0.82 0.10* 
Pre-Test 0.55 0.03 0.62** 0.28 0.05 0.25 
Missing Pre-Test Data -0.73 2.13 -0.01 -0.54 4.61 -0.01 
2 
R 48 0.16 
dl 28.68** 5.62 


* p<0.05; ** p<0.01 
Phonological Awareness. Three separate regressions, one predicting preschoolers’ 
phoneme manipulation skills, the second predicting initial sound phonological awareness, and 


the third measuring general phonological awareness (e.g., segmenting and blending word parts, 
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rhyming) were conducted to measure the impact of participating in UPSTART Reading are 


displayed in Table 8. Each regression included covariates to control for pre-test performance, 


missing pre-test data, and school districts. Results showed that UPSTART Reading enrolment 


had a unique and positive impact on phonemic manipulation (6 = .16, p = .000) and on initial 


word sounds phonological awareness (f = .15, p = .000), but not on general phonological 


awareness scale (6 = .07, p = .080) that measured the ability to blend, segment and rhyme words. 


Table 8: Regression Analysis of Predictors of Phonological Awareness (PA) 


Phoneme Manipulation Initial Word Sounds General PA 
(N=479) PA (N=496) (N=481) 
Variable B SEB B B SEB B B SEB B 
(Constant) 2.43 0.28 6.22 0.75 5.28 0.46 
Treatment 0.51 0.13 0.16** 1.38 0.36 0.15** 0.36 0.20 0.07 
Pre-Test 0.51 0.05  0.46** 0.47 0.04 0.49** 0.40 0.04 0.41 
See Pre- 9.52 0.54  -0.04 -6.98 o77 -0.10* 421i 0:76 ~ 20,67 
est Data 
R 25 31 21 
F 10.37*# 13.88** 8.22 


* p<0.05; ** p<0.01 


Decoding. In order to determine the unique impact of the treatment condition on 


children’s developing decoding capabilities, multiple regression analyses were conducted with 


decoding literacy scores as dependent variables and UPSTART Reading participation, decoding 


scores prior to beginning UPSTART, school district, and missing pre-test data as predictor 


variables. Results of the regression analyses displayed in Table 9 provide confirmation that 


UPSTART Reading had a significant positive impact on children’s emerging decoding skills of 


word recognition and reading common signs. The ability to read or recognize basic pre-primer 
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vocabulary words had a significant beta coefficient for the treatment predictor (6 = .24, p = 
.000), indicating that participation in UPSTART had a significant impact on children’s word 
recognition skills while controlling for prior knowledge, missing pre-test data, and school 
district. Similarly, assignment to the UPSTART Reading program had a significant contribution 
to children’s capacity to read or recognize common signs in their everyday environments (6 = 


11, p = .007). 


Table 9: Regression Analysis of Predictors of Decoding 


Word Recognition Common Signs 
(N=491) (N=475) 

Variable B SE B Bb B SEB Bb 
(Constant) 1.61 0.62 0.80 0.33 
Treatment 1.80 0.30 0.24** 0.43 0.16 Ooi 
Pre-Test 1.54 0.13 0.46** 0.89 0.07 0.20"* 
Missing Pre-Test Data 0.00 0.00 0.00 -1.30 1.01 -0.05 
Re 0.31 0.27 
ok 14.32 11.30 


* p<0.05; ** p<0.01 


Listening Comprehension. To test the hypothesis that participation in UPSTART 
Reading had a unique positive impact on preschoolers’ listening comprehension while 
controlling for prior knowledge, school district, and missing pre-test data, a regression analysis 
was performed. Results of the regression analysis did not confirm the hypothesis and indicated 
that participation in UPSTART did not have significant impact on children’s listening 


comprehension, / = -0.01, p = .763 (Table 10). 
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Table 10: Regression Analysis of Predictors of Listening Comprehension (N = 477) 


Variable B SE B pb 
(Constant) 14.05 0.68 

Treatment 0.08 0.26 0.01 
Pre-Test 0.25 0.03 0.34** 
Missing Pre-Test Data -3.18 1.28  -0.10 
R 31 

F 13.50" 


* p<0.05; ** p<0.01 


Vocabulary and Oral Language. A multiple regression was used to test the hypothesis 


that participation in UPSTART reading had a positive impact on children’s vocabulary and oral 


language skills was tested. Results displayed in Table 11 indicate that after controlling for pre- 


test scores, school district, and the presence of missing data, random assignment to UPSTART 


Reading did not have a significant impact on children’s vocabulary and oral language, / = -0.01, 


p= .681. 
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Table 11: Regression Analysis of Predictors of Vocabulary and Oral Language (N=482) 


Variable B SE B Bb 
(Constant) 17.06 1.01 

Treatment -0.17 0.42 -0.01 
Pre-Test 0.49 0.03 O57 ** 
Missing Pre-Test Data -0.95 1.53 -0.02 
R 43 

. 23.21** 


* p<0.05; ** p<0.01 


Effect Sizes. The effect size estimates are presented in Table 12 and show the magnitude 


of the average performance differences in standard deviation units between the treatment group 


and the control group on subscales measuring pre-literacy discrimination, letter knowledge, 


phonological awareness, decoding, vocabulary and oral language, and listening comprehension. 


Effect sizes were calculated based on the adjusted mean difference between the treatment and 


control groups divided by the unadjusted pooled standard deviation. The adjusted mean 


difference between the two groups was derived from the linear regression analysis and controlled 


for pre-test scores, school district, and the presence of missing data. 
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Table 12: Post-Test Effect Size Estimates based on Adjusted Means 


Construct Literacy Subscale Effect Size 
Pre-literacy skills Visual Discrimination 05 
Auditory Discrimination 02 
Phonological Awareness Initial Sounds Phonological Awareness 30** 
Phoneme Manipulation Joe 
General Phonological Awareness 14 
Alphabet Knowledge Identifies Uppercase Letters SL* 
Recites Alphabet 21* 
Decoding Word Recognition AQ** 
Reads Words from Common Signs oe 
Vocabulary and Oral Language Vocabulary and Oral Language -.03 
Listening Comprehension Listening Comprehension 02 


*p<.05, **p<.01, ***p<.001 


Overall, the UPSTART Reading program produced small to medium-size impacts on 
enhancing preschool children’s alphabet knowledge, as measured by the ability to recognize 
uppercase letters (Hedges’s g = .55) and recite the alphabet (Hedges’s g = .21). Similarly, 
participation in the UPSTART Reading program resulted in statistically significant positive 
effects in phonological awareness skills such as phoneme manipulation (Hedges’s g = .32) and 
identifying initial word parts and sounds (Hedges’s g = .30). UPSTART Reading generated 


small to medium-size effects on children’s emergent decoding abilities, as demonstrated with 


basic pre-primer vocabulary words (Hedges’s g = .49) and on signs commonly seen in children’s 


everyday environment (Hedges’s g = .22). 


Ze 


There were no significant differences between children assigned to the UPSTART 
Reading condition with those assigned to UPSTART Math on subscales that assessed pre- 


literacy discrimination, vocabulary and oral language, and listening comprehension. 


Discussion 

While there has been some research on the effects of educational technology on young 
learners (Foster, Erickson, Foster, Brinkman, & Torgesen, 1994; Mitchell & Fox, 2001), 
researchers acknowledge that the field is still in its infancy (Glaubke, 2007), and that available 
educational software typically is not formally evaluated for effectiveness in teaching reading 
skills (Wood et al., 2012). Adding to the body of research on educational technology and early 
literacy instruction, results from this randomized design study show that the UPSTART Reading 
program has significant effects on rural young children’s precursor literacy skills of phonological 
awareness (effect sizes ranging from .30 to .32), letter knowledge (effect sizes ranging from .21 
to .51), and decoding (effect sizes ranging from .22 to .49). 

Correlational studies have identified letter knowledge and phonological awareness as the 
two best predictors at school entry of how well children will learn to read during the first two 
years of instruction (National Reading Panel, 2000). As preschoolers enter kindergarten, the 
curriculum proceeds beyond letter recognition and phonological awareness to phonics 
instruction, which involves learning the grapheme-phoneme relationship, or the idea that letters 
correspond to sounds (Grant et al., 2012). Children who participate in UPSTART Reading during 
their preschool year are more likely to arrive at school with the prerequisite skills that contribute 
to future success. 

The positive impact of UPSTART on children’s letter knowledge, phonological 


awareness, and decoding skills stems from the program’s utilization of a phonics-based reading 


28 


program as part of its literacy curriculum which conforms to the National Reading Panel’s 
(2000) recommendation that early readers benefit from an explicit systematic approach to 
reading instruction. Moreover, UPSTART’s phonological awareness instruction adheres to the 
research-based practices outlined by Armbruster and colleagues at the National Institute for 
Literacy (2006), as it teaches children to use letters to manipulate phonemes as a precursor to 
decoding. 

There were, however, no significant differences between the treatment and control group 
after controlling for prior knowledge, missing pre-test data, and school district on subscales 
measuring listening comprehension and vocabulary/oral language skills, key components of 
deriving meaning from text. Other early reading interventions analogous to UPSTART have 
similarly shown larger impacts aligned to explicit code-based reading instruction (alphabet 
knowledge, phonemic awareness, decoding) than for comprehension (Simmons et al., 2011) 

It is possible that the UPSTART Reading curriculum failed to have an impact on 
children’s oral language, vocabulary, and comprehension above and beyond exposure to print- 
rich environments or the naturalistic home practices (e.g., actively reading stories to children, 
engaging in back and forth conversation) common in our participating families, regardless of 
experimental condition. Abrami, Borokhobski, and Lysenko (2015) note that vocabulary 
teaching, in particular, is influenced by the richness of the reading context and the motivational 
value of the text for the reader. 

The didactic presentation of vocabulary and e-book stories by the UPSTART program 
may also contribute to the lack of positive findings as other studies have pointed to the benefits 
of extended discussion between a preschooler and teacher during book reading sessions in 


building children’s vocabulary and comprehension (Dickinson & Smith, 1994; Gonzalez et al., 
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2011). It appears as though UPSTART’s benefits are more pronounced with the basic building 
blocks of reading that require direct and systematic instruction (i.e., recognizing letters, phoneme 
phonological awareness, decoding) as opposed to skills like vocabulary acquisition and oral 
language that are “hard wired” in children from birth and activated through reciprocal dialogue. 

Lastly, our comparison group consisted of children who participated in UPSTART Math, 
an educational software program that was identical to UPSTART Reading, save the curricular 
content. It is possible that enrollment in UPSTART Math conferred some benefits in early 
literacy to children, as a meta-analytic results of longitudinal school entry data sets indicate that 
early math skills were a most powerful predictor of later achievement in both math and reading 
(Duncan et al., 2007). 

The positive early literacy findings from children who participated in UPSTART Reading 
support previous research that suggest that rural students can benefit from focused and targeted 
literacy intervention (Amendum, Vernon-Feagans, & Ginsberg, 2011). More than 20 percent of 
all public school students in the United States are enrolled in rural school districts (Johnson, 
Showalter, Klein, & Lester, 2014), but these children are often overlooked in discussions of 
educational policy (Beeson & Strange, 2000). High-quality early education technology programs 
such as UPSTART that are readily accessible by families can be a viable pathway for preparing 
children in underserved areas to meet the demands of kindergarten and close the school readiness 
gap between rural children and their non-rural counterparts. 

Previous research of computer-based instructional programs has shown that benefits are 
linked with sufficient program use (Macaruso & Rodman, 2011) and additional analysis is 
needed to determine if outcomes are stronger for students who meet the requirements for 


minimum program use of UPSTART Reading compared to children who do fulfill the 
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recommended program use. Further research of the UPSTART Reading program will explore 
more nuanced questions, including the role of other factors such as preschoolers’ motivation and 
engagement with the program, parental support, and general home literacy environment that may 
mediate or moderate the relationship between UPSTART Reading participation and early 


literacy outcomes. 
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Appendix A 


Full Regression Tables 


Table A.1 


Regression Analysis of Predictors of Visual Discrimination (N = 482) 


Variable B SEB Bp 
(Constant) 13.75 0.54 
Treatment 0.15 0.25 0.03 
Pre-Test 0.20 0.02 0.38 
Missing Pre-Test Data -0.95 0.88  -0.05 
District 1 0.87 0.58 0.11 
District 2 0.94 0.64 0.09 
District 3 -1.33 0.76  -0.09 
District 4 1,51 0.81 0.09 
District 5 -0.21 0.68 -0.02 
District 6 0.19 0.67 0.02 
District 7 0.36 0.62 0.04 
District 8 0.18 0.94 0.01 
District 9 -0.04 0.64 -0.00 
District 10 -0.64 0.72  -0.05 
District 11 0.74 0.60 0.08 
District 12 17 0.64 0.13* 
Re 0.22 

F 8.85 


* p<0.05; ** p<0.01 
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Table A.2 


Regression Analysis of Predictors of Auditory Discrimination (N = 481) 


Variable B SEB Bp 
(Constant) 6.67 0.54 
Treatment 0.05 0.22 0.01 
Pre-Test 0.23 0.05 0.23 
Missing Pre-Test Data -3.48 0.81 -0.19 
District 1 -0.62 0.51 -0.09 
District 2 0.46 0.57 0.05 
District 3 -0.76 0.68  -0.06 
District 4 1.07 0.74 0.08 
District 5 0.94 0.61 0.09 
District 6 -1.04 059 -0.11 
District 7 -0.67 0.55  -0.08 
District 8 -2.09 0.84  -0.12* 
District 9 -0.41 0.57 = -0.05 
District 10 -0.19 0.63 -0.02 
District 11 -0.14 0.53  -0.02 
District 12 -0.24 0.57  -0.03 
R? 0.14 

F 5.00 


* p<0.05; ** p<0.01 
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Table A.3 


Regression Analysis of Predictors of Uppercase Letter Knowledge (N=480) 


Variable B SE B Bp 
(Constant) 10.80 Fats) 

Treatment 4.04 0.54 025"* 
Pre-Test 0.55 0.03 0:62** 
Missing Pre-Test Data -0.73 2.13 -0.01 
District 1 0.80 127 0.04 
District 2 -0.17 1.39 -0.01 
District 3 -1.43 1.72 -0.04 
District 4 0.66 1.81 0.02 
District 5 220 1.49 0.07 
District 6 U2 1.45 0.02 
District 7 2.77 1.36 0.10* 
District 8 1.58 2.06 0.03 
District 9 -1.25 1.40 -0.04 
District 10 -0.33 1.55 -0.01 
District 11 0.33 1.30 0.01 
District 12 4.06 1.40 0.14** 
‘Re 48 

i 28.68** 


* p<0.05; ** p<0.01 


4] 


Table A.4 


Regression Analysis of Predictors of Recites Alphabet (N = 476) 


Variable B SEB Bp 
(Constant) M25 abcess 
Treatment 1.97 0.82 0.10* 
Pre-Test 0.28 0.05 0.25 
Missing Pre-Test Data -0.54 461 -0.01 
District 1 4.45 L95- 2O.07* 
District 2 6.30 2.14 0.19** 
District 3 -4.69 2.59  -0.10 
District 4 Ash, 2.77 0.07 
District 5 5.90 2.34 0.15* 
District 6 4.19 223° O12 
District 7 4.42 2.10 0.14% 
District 8 2.88 3.17 0.05 
District 9 7.86 2.15 0.23 
District 10 -0.66 2.40  -0.02 
District 11 3.43 2.02 0.12 
District 12 TS 216° (0,22°* 
R? 0.16 

F 5.62 


* p<0.05; ** p<0.01 


42 


Table A.5 


Regression Analysis of Predictors of Phoneme Manipulation (N = 479) 


Variable B SEB Bp 
(Constant) 2.43 0.28 
Treatment 0.51 0.13 0.16** 
Pre-Test 0.51 0.05 0.46** 
Missing Pre-Test Data -0.52 0.54 -0.04 
District 1 0.06 0.30 0.01 
District 2 -0.03 0.33 -0.01 
District 3 -0.29 0.41 -0.04 
District 4 -0.04 0.43 -0.01 
District 5 -0.17 0.36 -0.03 
District 6 -0.17 0.35 -0.03 
District 7 -0.30 0.33 -0.06 
District 8 -0.29 0.49 -0.03 
District 9 0.29 0.33 0.05 
District 10 0.16 0.37 0.02 
District 11 0.20 0.31 0.04 
District 12 -0.56 0.34 -0.10 
R? 2) 

F OP bibad 


* p<0.05; ** p<0.01 
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Table A.6 


Regression Analysis of Predictors of Initial Word Sounds Phonological Awareness (N = 496) 


Variable B SEB Bp 
(Constant) 6.22 0.75 

Treatment 1.38 036-0, 1S** 
Pre-Test 0.47 0.04 0.49** 
Missing Pre-Test Data -6.98 2.77. -0.10* 
District 1 -0.14 0.84 -0.01 
District 2 3.18 0.92 0.20 
District 3 0.41 1.14 0.02 
District 4 -0.14 1.21  -0.01 
District 5 2.10 1.01 0.11% 
District 6 1.65 0.96 0.10 
District 7 235 0.91 0.15 
District 8 -0.76 1.37 = -0.03 
District 9 -0.67 0.93 -0.04 
District 10 -1.48 1.04  -0.07 
District 11 1.11 0.87 0.08 
District 12 1.69 0.93 0.10 
R2 fa 

F 13.88** 


* p<0.05; ** p<0.01 


Table A.7 


Regression Analysis of Predictors of General Phonological Awareness (N = 481) 


Variable B SEB Bp 
(Constant) 5.28 0.46 

Treatment 0.36 0.20 0.07 
Pre-Test 0.40 0.04 0.41 


Missing Pre-Test Data 
District 1 
District 2 
District 3 
District 4 
District 5 
District 6 
District 7 
District 8 
District 9 
District 10 
District 11 
District 12 
R? 


PF 


-1.21 0.76 = -0.67 
-0.74 048  -0.11 
0.67 0.53 0.08 
-1.95 0.64  -0.16** 
-0.36 0.69  -0.03 
-0.54 0.56 -0.06 
-1.72 0.55 -0.19** 
-0.65 0.52  -0.08 
2d? O.78- «0 Ast* 
-1.46 0.53 -0.17** 
-1.29 0.59  -0.12* 
-0.70 0.49 -0.10 
-0.46 0.54 = -0.05 


8.22 


* p<0.05; ** p<0.01 
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Table A.8 


Regression Analysis of Predictors of Word Recognition (N = 491) 


Variable B SEB Bp 
(Constant) 1.61 0.62 
Treatment 1.80 0.30 0.24** 
Pre-Test 1.54 0.13 0.46** 
Missing Pre-Test Data 0.00 0.00 0.00 
District 1 0.03 0.71 0.00 
District 2 -1.30 0.78  -0.10 
District 3 -1.89 0.96  -0.10* 
District 4 0.21 1.01 0.01 
District 5 0.03 0.85 0.00 
District 6 0.78 0.81 0.05 
District 7 -0.63 0.77 — -0.05 
District 8 “1.25 1.15 -0.05 
District 9 -1.30 0.78 = -0.10 
District 10 0.50 0.88 0.03 
District 11 -0.26 0.74 -0.02 
District 12 -0.29 0.79  -0.02 
R? 0.31 

F 14.32 


* p<0.05; ** p<0.01 
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Table A.9 


Regression Analysis of Predictors of Common Signs (N = 475) 


Variable B SEB Bp 
(Constant) 0.80 0.33 
Treatment 0.43 0.16. 0.114" 
Pre-Test 0.89 0.07 0.50** 
Missing Pre-Test Data -1.30 1.01  -0.05 
District 1 0.29 0.38 0.05 
District 2 0.08 0.41 0.01 
District 3 -0.46 0.51  -0.05 
District 4 0.04 0.54 0.00 
District 5 0.27 0.45 0.03 
District 6 -0.06 0.43 -0.01 
District 7 0.44 0.41 -0.07 
District 8 0.42 0.61 0.03 
District 9 0.10 0.42 0.01 
District 10 -0.11 0.46 -0.01 
District 11 0.33 0.39 0.06 
District 12 -0.26 0.42  -0.04 
R? 0.27 

F 11.30 


* p<0.05; ** p<0.01 
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Table A.10 


Regression Analysis of Predictors of Listening Comprehension (N = 477) 


Variable B SEB Bp 
(Constant) 14.05 0.68 
Treatment 0.08 0.26 0.01 
Pre-Test 0.25 0.03 0.34** 
Missing Pre-Test Data -3.18 1.28 -0.10 
District 1 -1.26 0.61 -0.14* 
District 2 3.05 0.67 0.26** 
District 3 -0.76 0.81  -0.05 
District 4 1.08 0.88 0.06 
District 5 pee 0.73 0.17** 
District 6 -1.26 0.70 -0.10 
District 7 1.66 0.66 0.15* 
District 8 1.36 0.99 0.06 
District 9 eA | 0.67 0.19** 
District 10 0.63 0.75 0.04 
District 11 0.97 0.63 0.10 
District 12 0.42 0.67 0.04 
R? 1 

F 13.50** 


* p<0.05; ** p<0.01 
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Table A.11 


Regression Analysis of Predictors of Vocabulary and Oral Language (N=482) 


Variable B SEB Bp 
(Constant) 17.06 1.01 

Treatment -0.17 0.42 -0.01 
Pre-Test 0.49 0.03 OST 
Missing Pre-Test Data -0.95 = 1.53 -0.02 
District | -3.96 0.98 -0.24** 
District 2 -2.19 1.08 -0.11* 
District 3 -2.91 1.29 -0.10* 
District 4 1.27 1.41 0.04 
District 5 -2.18 1.18 -0.09 
District 6 =3,70-- 4,93 -0.17** 
District 7 -0.84 1.07 -0.04 
District 8 -1.75 1.56 -0.05 
District 9 -5.57 1.09 -0.27** 
District 10 -3.69 = 1.21 -0.14** 
District 11 -1.71 1.02 -0.10 
District 12 -3.06 1.09 -0.15 
R 43 

si 23.21** 


* p<0.05; ** p<0.01 
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